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Whether there is a difference in the performance of 
public school and private school students on the advanced placement 
(AP) tests of the College Board was studied. The sample was composed 
of 216 public high schools, 32 Catholic high schools, and 24 other 
private high schools in Northern California, all of which gave their 
students at least five AP examinations. Subsamples were used to 
compare schools in the same locality. In the sample as a whole, 
private school students slightly (but not significantly) outperformed 
public school students. In a subsample of 148 schools, comprised of 
44 schools with highly developed preparation programs and 104 schools 
which might compete for students with these schools. Catholic school 
students were outperformed by public school students, and public 
school students were outperformed by other private school students. 
Catholic schools were less likely than other private schools to have 
highly developed programs of preparation for the tests. This study 
demonstrates that the sector the school occupies has no significant 
influence on this aspect of student performance and illustrates the 
importance of the quality of the school's program on performance on 
AP tests. Four tables give information on students test results. 
(SLD) 
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"There is general consensus that on the average. Catholic high 
school students outperform public hich school students on 
acnieveinent tests." — - Edward Haert^l (1987, p. 9) 

This statement, found at the beginning of HaertPl's chaptpr, 
reflects, if not the conventional wisdom, at least the opinion of 
the rr.any parents who, although not Catholic, hcve opted to send 
their children to Catholic schools (Cole.nan, Hoffer, and Kilgore, 
1931; Coleman and Hoffer, 1937; Schneider and Slaughter, 1983). 
Is this perception, based in large part on the highly publicizpd 
studies by Coleman, Hoffer and Kilgore (1981, 1982a) and Hoffer, 
Greeley, and Coleman (1935), accurate? 

The analyses of Cole.Tan, Hoffer and Kilgore, using the High 
Scnool and Beyond (HS3) base year data, and, to a lesser extent, 
those of Hoffer, Greeley, and Colenian, using the follow-up data, 
iiave oeen dissected by those skilled in statistical analysis. 
However, "different investigators' interpretations [have] ranged 
all the way fron. the position that overall sector differences 
[are] negligible to the position that there [is] a substantial 
Catholic sector advantage with strong implications for 
educational po: icy. [Furthermore,] rather than supporting sorr.e 
of these conclusions and refuting others, it appears that ^he 
follow-up data have essentially confirmed the earlier positions 
of everyone using th- new daca to test their original findings." 
(Haertel, Jai.ies, and Levin, o.o) 

••luc-h of the discussion about the various analyses of the hsb 
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dcta has revolved either about the treatment of backqround 
vdiicibles or about what constitutes gonuine significance. 
(HaercGl, 1987; Alexander, 1987; both include reviews of other 
studies). ilowever, the question of the validity of the HS3 test 
battery as a measure of sophomore to senior academic growth has 
also been raised (Heyns and Hilton, 1982; Goldberger and Cain, 
1932 ; Cain and Goldberger, 1983; Haertel, 1937; Will^ns, 1934 , 
1937). willr,-,s A-ritos: "iJone of the High School and Beyond t--sts 
■^pjpars to be an adequate m-asure of acadei-nic growth during the 

intervention period if the tests had measured more advanced 

skills in biology, physics, ch-a,istry, mathematics and English, 
then we r.^icjht have observed a si-,nif icant private schooling 
effect. At present, this must re:nain an open question (1937, 
Pi.. 129-130) ." [1] 

Providing a partial answer to that open question is the 
purpose of this study. it asks if there is a difference in the 
performance of public and privaf- school students on one of the 
acnieve.nent test batteries given annually to thousands of 
Ah.erican high school students, the advanced placement (AP) tests 
administered by the College Board. If private school students 
perfor.ii better than public school students on these tests, the 
findings of Coleman, Hoffer, and Kilgore and of Hoff-^r, Gieploy, 
•and Col".„an will have be^n given an independent boost. if, on 
u-.^- nt:ier hand public stu«Jfc-.-ts scorp as well as private school 
i.*--udents, it would indicate that on this diT.ension sectoi is not 
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a crucial factor in deteriTiinincj student outcones. 



Method 

3 a r;. o 1 

Tae sample coinprises the 272 Northern Ca 1 if ornia [ 2] high 
schools which, in the spring of 1987, gave at least five advanced 
placer.,ent -xafiis.[3] The r^ajority of these institutions, 216, are 
public; 32 are Catholic; and 24 are other private schools. A 
su.-nary of their A? examination scores was obtained. For each 
school and for each of the 26 exams administered in 1937, this 
su.T.Mdry consists of the nu.-nber of exams scored 1, 2, 3, 4, and 5, 
respectivf^ly . [4] 

Because the test is one that students normally take at the 
er.d of a year of study, individual data about the students is not 
collected. Furthermore, aggregate student data is unavailable 
for .nost of the schools. [5] However, the nature of the research 
makes it desireable for student background influences to be 
considered. Tnis was acc-omplished in two ways. First, 
subsamples ^ere specified to control partially for background 
factors. Second, in a random subsar.iple of 12 schools, the 
procedures for admitting students to advanr-oQ placement classes 
^er--^ rhpc-kpd for comparability. 
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Designed to compare public schools with the privatp schools 
likely to compete with them for students, subsample p/p consists 
of 201 schools: all 56 private schools and tho 145 public schools 
located vvithin 15 mil^s of at Ipast one of the private schools. 

Previous research (Bodenhausen , 1988) indicated that the 
nJKber of exa;r,s -jiven by a school is significantly related to its 
studc^nts* test scores. Subsample HDPl -.vas designed to compare 44 
sc!:ools with highly developed programs. The 26 public schools 
gave at least 125 exams; the 19 private schools (7 Catholic, 11 
other private) gave at least 50. These nunbers were chosen as a 
reasonable nuxber of exams to expect from a school with at least 
5 A? class-^s, assu.T.ir.g an aver^jo class size nf 25 in public 
schools and 12 in private ones. [6] 

In order to look at schools likely to compete with the 
schools in HDPl, subsamplp HDP2 was defined consisting of the 
schools in HDPl, public schools located within 15 miles of a 
private school in HDPl, and private schools within th« same 
distance of a public school in HDPl. Of th° 143 schools in HDP2 , 
102 are public, 25 are Catholic, and 21 are oth^r private. [7] 



The assign: 'ent p rocedures survey 

The AP coordinators at t.-;elve randomly selpc-ted schools werp 
contact-^d and asked how students are adr.iitted to their AP r.ath, 
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English, science, foreign language, and American history 
classes. while the responses take individual school conditions 
into consideration, they are .-markedly sir.ular. For AP classes in 
••vhich success is dependent upon writing ability, prospective 
students are required to de,.,onstrate appropriate writing and 
analysis skills before being per,nitted to enroll. For AP classes 
in .-hich success is dependent upon :.astery of prerequisite 
classes, verification {beyond just a passing grade) of that 
:nastery is required (usually by a combination of teacher 
rer-o,u-.endation, exa.i,, and interview). Because, according to all 
of the coordinators, their schools' procedures are bas-d upon 
recomr^endations from the College Board, it is likely that other 
schools also use comparable procedures. 



Variables 

Dichoto.Tious variables w«re defined specifying sector and 
subsa.Tiple inclusion. 

The endogenous variables of the study were defined as the 
percentage of exams scored 3, 4, or 5 (3^) for the school as a 
whole and for each of the following exa.TS (if adaimistered) : 
A..-,-rican history, biology, chemistry, English language and 
co,Too.sition, English literature and composition, French language, 
calculi^s AB, calculus 3C, and Spanish language. 



- 5 - 



ERIC 



While average exam scores were available for c^^^^i 
for eacn subject, chey were not used in the analysis because they 
are more influenced by student ability than are passing 
scores. [3] 



Analysis and Results 



In the sample as a whole and in each subsatnple, the .neans of 
oach of the endogenous variables were tested for significant 
difference between sectors. Table l su.-.narizes these values. 



Insert Table 1 about her<^ 



For the sa.nple as a whole and for subsa.Tiple P/p, they did not 
^irfer significantly by sector. In both, the m^ans were very 
slightly higher for private schools and quite a bit higher for 
non-Catholic private schools. However, any desire to attach 
importance to the other-private figures should be tempered by the 
ract that 461 of those schools had highly developed orogra.vs 
whoreas only 22% of Catholic schools and 13% (12% in the sairiple 
us u whol-) of public schools fell into that category. 

:.'h-n the .leans for subsa.rples HDPl and nDP2 are compared 
with that of P/P, the difference is significant, for HDPl at the 



.01 level and for HDP2 at the .05 Ipvel. Furthermore, within 
:iDP2, public school means exceed those of private schools, 
although not those of non-Catholic private schools. This v^as 
true even though only about one-quarter of the public schools 
vere from HDPl whereas one-third of the Catholic schools and over 
half of the other private schools carae from that group. 

Correlation coefficients between the sector and the 
endogenous variables were computed. The results were similar. 
Except for subsainple HDP2, the first digit after the decimal 
point of the coefficient was "0;" for that subset, the first 
digit wds 1 or 2 with the sign indir-ating the very slight 
^dvapcage belonged to public schools. 

of exams given by the school were 
regressed on each of the endogenous variables. Table 2 presents 
a 3uir,;,tary. 



Insert Table 2 about here 



Tlie resulting equations indicate that sector is not significant 
in determining student exam scores. The number of exams given by 

^ K-., i,v^«c-vr-L, ncis a Significant (at .01 l^^vel) 

relationship to student performance on AP teste. 

Finally, analyses wer^ conductpd within the subject areas of 
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calculus (AB &BC),[9J American historv. bioln.y c^--^^"-^- 
English language and coinposition, English literature ana 
composition, French language, and Spanish language. iixcept for 
calculus, these were nieans analyses only. Because students 
^^ntering AP n>ath classes have comparable math backgrounds [ 10] and 
are unlikely to have taken any calculus prior to beginning its 
study in an AP class, [11] regressions were also conout^d for that 



ub j ect 



Table 3 presents a summary of means by subject 



a rea 



Insert Table 3 about here 



While the highest mean percent of students passing the exa.r, was 
found in other private schools for most subjects, a higher 
percent of public school students passed the math exams. There 
./ere only three instances of the sector mean differing 
significantly (at the .05 level) from the p/p r,ean: the English 
literature ::-,ean for other private schools was significantly above 
it; the French rr.ean for Catholic schools and the BC calculus mean 
for other private schools vere significantly below it. 

Regressions were calculated for a subsa.nple designated ilATH 
— all 3.hools in P/P which gave at least 5 exams in either AB or 
3C calculus [59.3 % of the public schools in P/P (86 schools); 
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^iv.->u u.i^ ^uu.iv/ixC s^^.iOOib in f/r' scnoois); b2.5l of the 

other private schools in P/P (15 schools)] — and for a subsa.iiple 
J^>sicjnated MATII/HDP2. Tablp 4 suxrnarizps the regressions- 



Insert Table 4 about here 



In both cases, the number of exa.TS was the only significant 
variable (at .01 Ipvel). 



Pi scussion 

Talbert (1933, p. 151) writes, "The current reform ir.ove.Tient 
m US puolic education is bolstered by claims that private 
schools do better than public schools. Docu.rents such as the 
ni-nly puolici^ed report of Cole.-^an, Iloffer and Kilgore (1931; 
s-^-^ also li)82a and 1932iD) have creditied th'^ private sector with 

superior acade.nic productivity " But he also notes (p. 137) 

chat "••.subsequent analyses tend to challenge the claim tnat 
private schools are academically superior to public schools, 
aft^r student selection and curricular placements are adeguat'^ly 
t..:^*r i'^to accounts •••" 

Coleman, Hoffer and Kilgore 's study used the HS3 test ^hich 
^•^•.^.pnasizpd basic skills. Their conclusions, as not^d above, have 



he^n challen>jed pi-i,r,arily on thp oasis the lack of con'tols for 
student selection and curriculai- placerr.ents . Ilo.v-evpr, some of 
their critics raised the inadpquacy of tiie HSB tpst as a .-neasure 
of a significant private schooling effect (s^p, for -xanplp, 
neyns i Hilton, 1932; v;illnr,s, 1984 ; & HaertPl, Ja.^iPs, i Lpvin, 
1S37). This study proviups evidpnce the concprns about the nature 
of th*- ilS5 test drp wpU founded. 

Potential confounding factors werp addressed. ;;hilf- student 
body information was unavailable for ,T,ost of the schools in the 
arpa, previous resparch providpd justification for the assumption 
that Such factors do not confound th=> study's results. 
Bodenhausen (1983), in her study of public high schools in thp 
sa.-, e area, found that studpnt body pffpcts have no significant 
vplationsnip to advancpd placprnent tPst scorps; Coler.ian, Hoffpr, 
and Kilgorp (1982) found privatp schools rnorp effpctive than 
public in xininizing the pffpct of diffprpnc^s in students' 
backcrounds. 

The assu.v,ption that the results wprp unconfounded by earlier 
schooling expot-ipnces vas supported by thp random survpy of 
ijchools' practicps in ad.iitting students to AP coursps. It './as 
also justified by the lack of spctor effect in ~a h^.^a 1 1 cs , an 
>i!-ea in which students arp unlikely to have had any previous 
sobjpct ..-.dtter exposure. 

In the San Franciaco Bay area Ahprp the majority of th" 
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schools constdprpd in this study arc located, almost all 
non-public secondary schools have selective admissions policips. 
B^-^CdUSP of this selectivity, one might pxpect the private scnool 
scuJents to outperform those fro.-n /-ublic schools on t-sts sur-h as 
the A? exar:TS. T!iis, ho//evpr, ^^ras not the case. 

Althou.jh in the sdi.iplp as a whole private school studpp.ts 
sli^.-itly, out rot significantly, outperformpc public school 
students, they did not do so in the suosampl ps . In both P/P and 
110P2, thp pprformancp of public school studpnts was slightly, but 
again not significantly, greater than that of private school 
students. When Catholic and non-Catholic private schnols wer^ 
considerpd spparately, a different picture Pmerged. Catholic 
school students w°re grpatly outperformpd by public school 
students; in HDP2, the differpncp was significant at thp .01 
iG^el. Public school studpnts were, >. turn, outperformed by 
"ot.^er private" school students; in FD?2, thp diffprer.cp was 
significant at the .05 level. m the latter case, thp 
dif f^'t-ential was probably influenced by the twice as grpat 
livelihood that an "other private" school in HDP2 had a highly 
devploped program than that a public school did. iiovever, sincp 
a Catholic school was nearly 10% more likely to have such a 
pro:>-a,n than a puol ic one, tr.at factor could not e-<p]7i\r. f-h^ 
relo'-v'^'ly poor perf or-nance of Catholic sr-hool students. 

further indication the t S'^^ctor •='ffects rray well b'=> soerious 
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is providf^d by the results of thp math pxarr^s. Here again, the 
top perf or.Tiers were the public school students. However, in BC 
■ calculus, the mean score of the other private school students was 
significantly below that of public and Catholic school students. 

Alexander writes (1937, p. 53), "I conclude that sector 
effects on cognitive perfor:nance are sn^all at best. In fact, I 
would not be at all surprised if they turned out to be entirely 
non-^^xistant when properly studied." This research corroborates 
that conjecture. 

This research also poipts to a factor in achieveT>ent which 

has bpen all but ignored m previous research prograr^.. In un 

area in which a large numoer of public and private 3-;hools 
cou^pete with each other for students, the question of -/hether 
able students go to a school because of its program or whether 
the progra^n has evolved to .-neet the needs of such students .Tiay be 
akin to that of whether the chicken or thp egg came first, 
iiowev^r, because of the significant difference in the achieverr.ent 
of s*:udents from scnools with ^.ighly developed progra.ns and that 
of students from other schools, the effect of progran .T.erits 
further investigation. 

Conclusion 

This study has implications for policy a^akers. Because it 

- 12 - 

ERIC 



shows that sector status has no significant influpncP on at least 
on^ aspect of student performance, it should raise -uestions for 
those w-ho support the use of public monies to expand private 
education. Furthermore, because it also demonstrates that the 
type of progrdTi offered by a school has a significant influence 
on the perforrr.ance of one group of students, it should spur 
interest in experiments which enablp a school to offer a richer 
program than it would otherwise be able to afford. 
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TABLE 1 

t'-an Percent of Students in School Taki 



:iean 



Aea n 



Mean 



Mean 



ng an AP Sxam Passing the Exam 
Overall Public Private 



5 5.3 

P/P 
69.0 



65.4 



63.4 



Rural 
59.0 



HDPl 
79. 6** 



P/P public P/p ra^-hnlir- d / n 

!r / r v.ci.noiic P/P otn'^r private 



59.1 



65 



71 



HDP2 HDP2 public HDP2 Catholic nDP2 other private 

^2. 3 66 . US 76. H 

P/P co;;,posite private HDP2 composite private 

^3.7 70.7 



tniz-i ?ro^ -el- - 

y ^iire.ent fro.n :IDP2 public value at .01 l^vel 
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TABLE 2 



Regression Results 

Pei-cent of Students Passing Exa-i Regressed or. Sector and 

Nua-.ber of 2xa;r.s Given 







Subsample p/p 








variable 


Coef f 


F-ratio 


Prob. 


Std. 


Error 


publ ic 
Cathol ic 
no. exams 


-2.01 
-2.24 
. 08 


. 25 5 
.217 
21.314* 


.614 
.647 
.000 


3. 30 
4.82 
.02 




* significant 


at .01 


1 eve 1 

Subsaniple HDP2 








var iable 


Coeff 


F-ratio 


Prob. 


Std. 


Error 


(jub 1 ic 
Catr ->1 ic 
no. <=>xa.'ns 


-3.75 
-7.72 
.06 


.958 
2.625 
15.273* 


.572 
. 103 
.00 0 


3.33 
4.75 
.02 





* significant at .01 level 
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TABLE 3 



IJpan Percent of Students in School Taking an AP E 
Passing thp Exam By Subject 





A:n . Hi s toi"y 


Biol ogy 


Che.Tiistry 


Frpncii 


P/P r.ioan 
(no. schools) 


C2.4 
(121) 


73.3 
(53) 


64.2 
(33) 


69.3 
(38) 


Public n^.pan 
(no. scl'iools) 


60.7 

(90) 


63.4 
(31) 


61.4 

(26) 


69.0 
(2G) 


Catholic mean 
(no. schools) 


61.3 

(IG) 


74.0 
(10) 


S3. 3 

(1) 


33.4* 
(4) 


Othf^r private m^an 
(no. schools) 


74 .0 

(15) 


37.7 

(12) 


66.5 

(5) 


85.7 

(S) 




E n y . 1 a n y . 


Enc;. lit. 


Spanish 




P/P mean 
(no. schools) 


64 . 0 

(39) 


68.4 

(144) 


80.7 
(71) 




Public mean 
(no. schools) 


60.6 

(28) 


55.9 
(112) 


30.4 

(53) 




Catholic* u.edn 
(no. scliools) 


79.7 

(5) 


72.5 

(19) 


68.9 

(9) 




Other private mean 
(no. schools) 


65.5 

(6) 


83.8* 
(13) 


94.0 

(9) 






Calculus A3 


Calculus 


BC 




P/P mean 
(no. schools) 


78. 1 

(104) 


76.9 

(34) 






Public mean 
(no. schools) 


78. 3 

(75) 


80.7 

(26) 






Catholic RiOan 
(no. schools) 


75. 4 

(15) 


80.0 
(1) 






Other private mean 
(no. schools) 


79. 9 

(13) 


62.0* 

(7) 






* significantly different frorr. p/p 


valup at .05 


levpl 




P/P sample size is 145 


public, 32 Catholic, 24 othpr privat 


e . 
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TABLE 4 

Regression Rpsclts for y.ath Subsainples 

?prc«=nt of Students Passing Exam Recressecl on Sector and 
j;un\ber of Exams Given -- Subsa.nple MATH 

'^^^i^^le Coeff. F-ratio Prob. Std. Error 

no. exarr.s .0023 13 .27* .000 . 0215 ("1.5 exams) 

The CoiT.puter computed coefficients for no other variables 

* significant at .01 level 

Percent of Students Passing Exam Regr«=Gsed on Sector and 
Number of Exams Given -- Subsaniple MATn/HDP2 

variable Coeff. F-ratio Prob. Std. Error 

r.o. exams . 0357 14 . 54* . 001 .0223 ("1.3 exams) 

Tlie Co., puter computed coefficients for no other variables 

* significant at .01 level 
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NOTES 



1. In the opinion of the researcher, the situation is even worse 
than that implied by Willms' statement. Three of the seven HSB 
tests were designed solely as measures of basic skills in general 
mathematics, reading, and vocabulary. The common items from 
those tests, the only ones used in CHK ' s base-year analysis , 
could easily have been taken from one of the high school 
proficiency tests, specified to be at the eighth grade level 
developed by California school districts. The other four tests, 
specified to be curriculum-specific in more advanced mathematics, 
science, writing, and civics, comprised 10, 20, 17, and 10 items, 
respectively. As a mathematics teacher, the researcher is amazed 
that a test with only 10 questions could be considered to cover 
"advanced" high school mathematics; she suspects that teachers in 
the other disciplines might feel similarly. 

2. Northern California specifically means those areas with zip 
codes beginning with 94, 95, and 96. Geographically, this region 
encompasses Santa Cruz and San Jose and areas to the north and 
east. 

3. The number, five, is in a sense arbitrary. in a random 
telephone sample of Northern California high schools which gave 
at least one AP exam in 1987, it was, however, the minimum number 
of exams given by any school which indicated that it had an 
actual AP program. 

4. Advanced placement exams are scored numerically from 1 to 5, 
5 high. Colleges usually give credit toward graduation for 
scores of 3 or better; thus, scores of 3, 4, and 5 are often 
termed "passing." 

5. The aggregate student data was collected as a part of a 
previous study (Bodenhausen, 1988). Relevant findings will be 
discussed in the Discussion section. 

6. Because in some, but not all Catholic schools, class sizes 
may equal those of public schools, this subsample could include 
some Catholic schools without highly developed programs. All of 
these are included in HDP2 by zipcode. 

7. It should be noted that there are high schools, both public 
and private, within the geographic area defined by HDP2 that are 
not in the subsample because they gave from 0 - 4 AP exams in 
1987. 

8. See the Discussion section for clarification of this point. 
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9. Two different AP calculus exams are offered. The AB exam 
covers approximately 1 semester of college calculus; the BC till 
covers between 2 quarters and 2 semesters. 

10. The precalculus math curriculum in the United States 
relatively standard: two years of algebra, one year of geomet^v 
a year of math analysis (the title of this course Varies) and 
Sird^ oTt-^e'^^rlhyr °^ ^ ^^luded in ^efthU ife 

i;;rod^;:t?^r^trthen\^^uluT^\^.-t tfache^Vere -t 

^e\%\^rTal^'2'Tchooir%'^^^'^' ^ surve^ of 'calculus 

teacners at 12 schools offering BC calculus. in none of the 

T'lL'^r": '^H^'"^' ^""^'^ ^^^^^1^^ t° enrolling 

AP. The researcher is aware that at one large Southern California 

anl ac'in^ih.fr calculus' in their "junior year 
h^frd^f^ht'p^^c™ ''''' California, no one'had 
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